HybridMingler: Towards Mixed-Reality Support for Mingling at Hybrid Conferences

Mingling, the activity of ad-hoc, private, opportunistic conversations ahead of, during, or after breaks, is an important socializing activity for attendees at scheduled events, such as in-person conferences. The Covid-19 pandemic had a dramatic impact on the way conferences are organized, so that most of them now take place in a hybrid mode where people can either attend on-site or remotely. While on-site attendees can resume in-person mingling, hybrid modes make it challenging for remote attendees to mingle with on-site peers. In addressing this problem, we propose a collaborative mixed-reality (MR) concept, including a prototype, called HybridMingler. This is a distributed MR system supporting ambient awareness and allowing both on-site and remote conference attendees to virtually mingle. HybridMingler aims to provide both on-site and remote attendees with a spatial sense of co-location in the very same venue location, thus ultimately improving perceived presence. © 2023 Owner/Author.

Engagement Estimation using Time-series Facial and Body Features in an Unstable Dataset

The global education sector has been deeply shaken by COVID-19 and forced to shift to an online teaching model. However, the lack of face-to-face communication and interaction in online learning is critical to high-quality teaching and learning. Research on engagement is a crucial part of solving this problem. Because engagement is of time-series data with an ongoing change, research datasets used for engagement analysis need a certain preprocessing method to capture time-series related engagement features. This research proposed a novel deep learning preprocessing method for improving engagement estimation using time-series facial and body information to restore traditional scenes in online learning environments. Such information includes head pose, mouth shape, eye movement, and body distance from the screen. We conducted a preliminary experiment on the DAiSEE dataset for engagement estimation. We applied skipped moving average in data preprocessing to reduce the influence of the extracted noises and oversampled the low engagement level data to balance the engaged/unengaged data. Since engagement is continuous and cannot be captured at a particular instant in time or single images, temporal video classification generally performs better than static classifiers. Therefore, we adopted long short-term memory (LSTM) and Quasi-recurrent neural networks (QRNNs)sequence models to train models and achieved the correct rate of 55.7% (LSTM) and 51.1% (QRNN) using the original key points extracted from OpenPose. Finally, we proposed the optimization structure network achieved the engagement estimation correct rate of 68.5% in proposed LSTM models and 64.2% in QRNN models. The achieved correct rate is 10% higher than the baseline in the DAiSEE dataset. © 30th International Conference on Computers in Education Conference, ICCE 2022 - Proceedings.

Quality Improvement: Efficacy of a Cardiopulmonary Rehabilitation Hybrid Program During the COVID-19 Pandemic

Background and/or Objectives: Our aim is to retrospectively evaluate the efficacy of our cardiopulmonary phase II hybrid program, which was created as an alternative to traditional, in-person phase II programs during the COVID-19 pandemic. Our hypothesis is that patients who enrolled and completed the hybrid program will demonstrate improvement in cardiopulmonary functional outcomes. Findings from this study will help inform project team members who can maintain or otherwise modify the hybrid program with the support of the PM&R Service Chief. Design(s): Our preliminary analysis included chart review and retrospective comparison of 22 patients' baseline and discharge 6-minute walk tests (6MWT), Duke Activity Status Index (DASI), weight, and exercise time. Pre-and post-rehab data will be compared with paired ttests. Level of significance (alpha) will be set at < 0.05. Additionally, age, sex, referring clinic, and primary/ secondary diagnoses were included for ongoing comparison. Setting(s): Veterans Affairs hospital for in-person appointments and virtual home-based setting for telerehabilitation component. Participant(s): Veterans who enrolled and completed the hybrid program between 9/1/2020-6/31/2021. Intervention(s): Retrospective chart review, interventions not applicable. Main Outcome Measure(s): Pre-and post-rehab functional outcome measures, including the 6-minute walk test and Duke Activity Status Index (DASI), will be compared. Secondary measures include weight change and exercise time following program completion. Result(s): In our preliminary data, 58% of patients demonstrated some level of improvement in the 6MWT. 50% achieved a favorable weight change and 64% showed improvement in the DASI. 100% of patients were able to increase duration of exercise time. A statistically significant difference (p < 0.05) was observed between pre- and post-rehab exercise tolerance time as well as the DASI. Conclusion(s): If further analyses are consistent with these findings, the results will guide future practice. Preliminary data indicate the hybrid program did not lead to statistically significant improvements in 6MWT or weight but did significantly improve exercise time and activity status level.

HCMUS MediaEval 2021: Multi-Model Decision Method Applied on Data Augmentation for COVID-19 Conspiracy Theories Classification

Corona Virus and Conspiracies Multimedia Analysis Task is the task in MediaEval 2021 Challenge that concentrates on conspiracy theories that assume some kind of nefarious actions related to COVID-19. Our HCMUS team performs different approaches based on multiple pretrained models and many techniques to deal with 2 subtasks. Based on our experiments, we submit 5 runs for subtask 1 and 1 run for subtask 2. Run 1 and 2 both introduces BERT[5] pretrained model but the difference between them is that we add a sentimental analysis to extract semantic feature before training in the first run. In run 3 and 4, we propose a naive bayes classifier[4] and a LSTM[8] model to diversify our methods. Run 5 ultilize an ensemble of machine learning and deep learning models - multimodal approach for text-based analysis[3]. Finally, in the only run in subtask 2, we conduct a simple naive bayes algorithm to classify those theories. In the final result, our method achieves 0.5987 in task 1, 0.3136 in task 2. Copyright 2021 for this paper by its authors.

Estimation of Learners’ Engagement Using Face and Body Features by Transfer Learning

In recent years, online learning plays an essential part in education due to distance learning technology development and control of COVID-19. In this context, engagement, a mental state to enhance the learning process, has been brought into the limelight. However, the existing engagement datasets are of a small scale and not suitable for education time-series research. We proposed an estimation method on time-series face and body features captured by built-in PC cameras to improve the engagement estimation on small and irregularly wild datasets. We designed upper body features using the facial and body key points extracted from OpenPose. To reduce the influence of the extracted noises from OpenPose, the moving average, the average value of a fixed period in the videos, is used to process the training data. Then, we compose a time-series dataset of online tasks with 19 participants. In the composed dataset, there remained self-reports of participants’ mental state and external observation to confirm the different engagement levels in the answering process. The combined self-reports and external observation results were used as the engagement label. Finally, the transfer learning was used to solve the insufficient data issue. We pre-trained a long short-term memory (LSTM) sequence deep learning model on a big dataset and transferred the trained model to share learned feature extraction and retrain our dataset. Our proposed method achieved 63.7% in experiments and could apply to estimate and detection engagement in future works. © 2021, Springer Nature Switzerland AG.